Method and apparatus for displaying and varying binocular image content

ABSTRACT

A controller that may implement variation of the content of binocular images which may depend upon which region of a binocular image a viewer is fixating. An aspect of the present disclosure may include locally controlling the viewer&#39;s perceived depth impression which may depend on where in perceived depth in an image the viewer is fixating. This may enable the perceived depth to be optimized across the image for quality and performance reasons.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationSer. No. 61/766,599, filed Feb. 19, 2013, entitled “Binocular fixationimaging method and apparatus,” the entirety of which is hereinincorporated by reference.

TECHNICAL FIELD

The present disclosure generally relates to image processing, and morespecifically, to depth budget and image processing methods, andtechnologies.

BACKGROUND

Depth budget has become an important concept in binocular imagecreation. It may create a limit on the total binocular effect in a threedimensional image. This limit in practice is determined by consideringmany factors including the limits of the human visual system and theparameters of the image display device being used to present the imageto the viewer. For stereoscopic images presented in an image plane thedepth budget is often discussed in terms of depth behind and in-front ofthe image plane.

Techniques are known for controlling the perceived depth in stereoscopicimages so that the total binocular effect remains within the depthbudget by controlling the capture or synthesis of the image(s). FIG. 15illustrates a reference U.S. Pat. No. 6,798,406, which generallyprovides a method for producing a stereoscopic image using at least onereal or simulated camera wherein the depth of a scene is mapped to apredetermined depth budget in the perceived stereoscopic image. FIG. 21illustrates a reference U.S. Patent Pub. No. US 2011/7,983,477, whichgenerally discusses variable depth mapping from scene to perceivedstereoscopic image. In addition a method such as that disclosed in U.S.Pat. No. 8,300,089 can also be used for variable depth mapping in thedepth (Z) dimension.

The eye's binocular fixation may be determined using a range of eyetracking devices, either by tracking both eyes or by tracking a singleeye and inferring the other from this information. One example of abinocular fixation tracking system is the Eyelink 1000, by ResearchLtd., Mississauga, Ontario, Canada, which tracks both eyes at highspeed.

BRIEF SUMMARY

An aspect of the present disclosure provides a controller thatimplements variation of the content of binocular images depending uponwhich region of a binocular image a viewer is fixating. An aspect of thepresent disclosure includes locally controlling the viewer's perceiveddepth impression depending on where in perceived depth in an image theviewer is fixating. This has the benefits of enabling the perceiveddepth to be optimized across the image for quality and performancereasons.

According to an aspect of the disclosure, a binocular imaging system mayinclude a display for presenting a left eye image and a right eye imageperceptually simultaneously, in which the left eye image has anassociated left eye field of view of the display and the right eye imagehas an associated right eye field of view of the display. A gazetracking element may also be included that may identify at least one orboth gaze directions of the left eye and the right eye. The binocularimaging system may further include an image controller that maycalculate a binocular region of fixation for the left and right eye, andthat alters the displayed left and right eye images. The imagecontroller may alter a subsequently displayed binocular image inresponse to a change in the region of binocular fixation between acurrently displayed binocular image and the subsequently displayedbinocular image. Altering the displayed left and right eye images mayaffect the local image depth content in the binocular region of fixationand surrounding the binocular region of fixation. The binocular regionof fixation may include a three dimensional region in which the locationvaries with the gaze direction of one or both of the left and righteyes.

According to another aspect of the disclosure, a method for varyingbinocular image content may include displaying a current binocularimage, and using input from the current binocular image, informationfrom a gaze tracker and scene depth measurement information to calculatea region of binocular interest (RBI) in a scene. The method may alsoinclude determining whether the region of binocular interest has changedand calculating the scene depth range for mapping to the depth budgetwhen the region of binocular interest has changed. The method mayinclude using a camera control algorithm to generate a subsequentlydisplayed binocular image using the scene depth range and making thecurrently displayed image, the subsequently displayed binocular image.

The method for varying binocular image content may further includereceiving a second input from the gaze tracker and scene depth measureand using the second input from the current binocular image, the gazetracker and the scene depth measure to calculate the region of binocularinterest in the scene when the region of binocular interest has notsubstantially changed. The method may also include determining a regionof binocular fixation in display space (RBF_(d)) by using gaze trackinginformation from a viewer watching a displayed binocular image andcalculating the equivalent region of binocular fixation in a scene space(RBF_(s)) by using the region of binocular fixation in display space(RBF_(d)) provided an image controller. In determining whether theregion of binocular interest has changed, the method may include usingthe region of binocular fixation in display space (RBF_(d)) and theequivalent region of binocular fixation in the scene space (RBF_(s)).The method may further include changing the region of binocular interestbased on scene changes while the region of binocular fixation in displayspace does not substantially change.

According to another aspect of the disclosure, a method for varyingbinocular image content may include displaying a current binocularimage, using input from the current binocular image and a gaze trackerto calculate a subsequent region of binocular fixation, and determiningany change in binocular fixation between a current region binocularfixation and the subsequent region of binocular fixation. If the case ofa change in binocular fixation between a current region binocularfixation and the subsequent region of binocular fixation, the method mayinclude calculating a disparity range of the subsequent range ofbinocular fixation. The method may also include determining whether thedisparity range is substantially zero and creating a subsequentlydisplayed image when the disparity range is not substantially zero. Themethod may also include making the currently displayed image, thesubsequently displayed binocular image.

Continuing the discussion, the method may include receiving a secondinput from the gaze tracker and using the second input from the currentbinocular image and the gaze tracker to calculate a subsequent region ofbinocular fixation when the subsequent region of binocular fixation hasnot substantially changed. The method may include receiving a thirdinput from the gaze tracker and using the third input from the currentbinocular image and the gaze tracker to calculate a subsequent region ofbinocular fixation when the disparity range is approximately zero. Thegaze tracker may determine the disparity within the fixated region, inwhich the gaze tracker determines the plane of fixation from thedifference between left eye and right eye screen fixation points.

In the case the method determines whether the disparity range issubstantially zero, the method may include comparing the image disparityof the subsequent object with zero, in which the subsequent object isbeing imaged where it is the closest object to a viewer in the region ofbinocular fixation. The method may also include altering a subsequentlydisplayed image in response to a change in the region of binocularfixation between the currently displayed binocular image and thesubsequently displayed binocular image and also may form a currentlydisplayed binocular image. Forming a currently displayed binocular imagemay include estimating a 3D region of fixation and projecting the 3Dregion of fixation into an image plane to form a binocular region offixation. The currently displayed binocular image is formed as a leftimage and a right image and may be selected from a larger source image.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are illustrated by way of example in the accompanyingfigures, in which like reference numbers indicate similar parts, and inwhich:

FIG. 1 is a schematic diagram illustrating one embodiment of is abinocular imaging apparatus, in accordance with the present disclosure;

FIG. 2 is a schematic diagram illustrating one embodiment of images forthe left and right eye, in accordance with the present disclosure;

FIG. 3 is a schematic diagram illustrating one embodiment of a currentlydisplayed binocular image pair in accordance with the presentdisclosure;

FIG. 4 is a schematic diagram illustrating one embodiment of theviewer's region of binocular fixation, in accordance with the presentdisclosure;

FIG. 5 is a schematic diagram illustrating one embodiment of an adisplayed image with little to no disparity, in accordance with thepresent disclosure;

FIG. 6 is a schematic diagram illustrating a flow chart, in accordancewith the present disclosure;

FIG. 7 is a schematic diagram illustrating one embodiment of a binocularimage pair, in accordance with the present disclosure;

FIG. 8 is a schematic diagram illustrating one embodiment of a displayedimage, in accordance with the present disclosure;

FIG. 9 is a schematic diagram illustrating one embodiment of a displayedimage, in accordance with the present disclosure;

FIG. 10 is a schematic diagram illustrating one embodiment of adisplayed image, in accordance with the present disclosure;

FIG. 11 is a schematic diagram illustrating a flow chart, in accordancewith the present disclosure;

FIG. 12 is a schematic diagram illustrating one embodiment of a gazetracking system, in accordance with the present disclosure;

FIG. 13 is a schematic diagram illustrating one embodiment of a scenespace, in accordance with the present disclosure;

FIG. 14 is a schematic diagram illustrating a flow chart, in accordancewith the present disclosure;

FIG. 15 is a schematic diagram illustrating one embodiment of abinocular image, in accordance with the present disclosure;

FIG. 16 is a schematic diagram illustrating one embodiment of a scenedepth range and depth budgets, in accordance with the presentdisclosure;

FIG. 17 is a schematic diagram illustrating one embodiment of an imagecontroller's response, in accordance with the present disclosure;

FIG. 18 is a schematic diagram illustrating one embodiment of an imagecontroller's response, in accordance with the present disclosure;

FIG. 19 is a schematic diagram illustrating one embodiment of an imagecontroller's response, in accordance with the present disclosure;

FIG. 20 is a schematic diagram illustrating a flow chart, in accordancewith the present disclosure;

FIG. 21 is a schematic diagram illustrating one embodiment of scenedepth range and perceived depth range, in accordance with the presentdisclosure;

FIG. 22 is a schematic diagram illustrating one embodiment of scenedepth range and perceived depth range, in accordance with the presentdisclosure;

FIG. 23 is a schematic diagram illustrating one embodiment of scenedepth range and perceived depth range, in accordance with the presentdisclosure;

FIG. 24 is a schematic diagram illustrating one embodiment of scenedepth range and perceived depth range, in accordance with the presentdisclosure;

FIG. 25 is a schematic diagram illustrating one embodiment of scenedepth range and perceived depth range, in accordance with the presentdisclosure;

FIG. 26 is a schematic diagram illustrating a flow chart, in accordancewith the present disclosure;

FIG. 27 is a schematic diagram illustrating one embodiment of scenedepth range and perceived depth range, in accordance with the presentdisclosure;

FIG. 28 is a schematic diagram illustrating one embodiment of a scenedepth range and perceived depth budget; and

FIG. 29 is a schematic diagram illustrating one embodiment of an imagesystem, in accordance with the present disclosure.

DETAILED DESCRIPTION

Generally, according to one aspect of the disclosure, a binocularimaging system may include a display for presenting a left eye image anda right eye image perceptually simultaneously, in which the left eyeimage has an associated left eye field of view of the display and theright eye image has an associated right eye field of view of thedisplay. A gaze tracking element may also be included that may identifyat least one or both gaze directions of the left eye and the right eye.The binocular imaging system may further include an image controllerthat may calculate a binocular region of fixation for the left and righteye, and that alters the displayed left and right eye images. The imagecontroller may alter a subsequently displayed binocular image inresponse to a change in the region of binocular fixation between acurrently displayed binocular image and the subsequently displayedbinocular image. Altering the displayed left and right eye images mayaffect the local image depth content in the binocular region of fixationand surrounding the binocular region of fixation. The binocular regionof fixation may include a three dimensional region in which the locationvaries with the gaze direction of one or both of the left and righteyes.

According to another aspect of the disclosure, a method for varyingbinocular image content may include displaying a current binocularimage, and using input from the current binocular image, informationfrom a gaze tracker and scene depth measurement information to calculatea region of binocular interest (RBI) in a scene. The method may alsoinclude determining whether the region of binocular interest has changedand calculating the scene depth range for mapping to the depth budgetwhen the region of binocular interest has changed. The method mayinclude using a camera control algorithm to generate a subsequentlydisplayed binocular image using the scene depth range and making thecurrently displayed image, the subsequently displayed binocular image.

The method for varying binocular image content may further includereceiving a second input from the gaze tracker and scene depth measureand using the second input from the current binocular image, the gazetracker and the scene depth measure to calculate the region of binocularinterest in the scene when the region of binocular interest has notsubstantially changed. The method may also include determining a regionof binocular fixation in display space (RBF_(d)) by using gaze trackinginformation from a viewer watching a displayed binocular image andcalculating the equivalent region of binocular fixation in a scene space(RBF_(s)) by using the region of binocular fixation in display space(RBF_(d)) provided an image controller. In determining whether theregion of binocular interest has changed, the method may include usingthe region of binocular fixation in display space (RBF_(d)) and theequivalent region of binocular fixation in the scene space (RBF_(s)).The method may further include changing the region of binocular interestbased on scene changes while the region of binocular fixation in displayspace does not substantially change.

According to another aspect of the disclosure, a method for varyingbinocular image content may include displaying a current binocularimage, using input from the current binocular image and a gaze trackerto calculate a subsequent region of binocular fixation, and determiningany change in binocular fixation between a current region binocularfixation and the subsequent region of binocular fixation. If the case ofa change in binocular fixation between a current region binocularfixation and the subsequent region of binocular fixation, the method mayinclude calculating a disparity range of the subsequent range ofbinocular fixation. The method may also include determining whether thedisparity range is substantially zero and creating a subsequentlydisplayed image when the disparity range is not substantially zero. Themethod may also include making the currently displayed image, thesubsequently displayed binocular image.

Continuing the discussion, the method may include receiving a secondinput from the gaze tracker and using the second input from the currentbinocular image and the gaze tracker to calculate a subsequent region ofbinocular fixation when the subsequent region of binocular fixation hasnot substantially changed. The method may include receiving a thirdinput from the gaze tracker and using the third input from the currentbinocular image and the gaze tracker to calculate a subsequent region ofbinocular fixation when the disparity range is approximately zero. Thegaze tracker may determine the disparity within the fixated region, inwhich the gaze tracker determines the plane of fixation from thedifference between left eye and right eye screen fixation points.

In the case the method determines whether the disparity range issubstantially zero, the method may include comparing the image disparityof the subsequent object with zero, in which the subsequent object isbeing imaged where it is the closest object to a viewer in the region ofbinocular fixation. The method may also include altering a subsequentlydisplayed image in response to a change in the region of binocularfixation between the currently displayed binocular image and thesubsequently displayed binocular image and also may form a currentlydisplayed binocular image. Forming a currently displayed binocular imagemay include estimating a 3D region of fixation and projecting the 3Dregion of fixation into an image plane to form a binocular region offixation. The currently displayed binocular image is formed as a leftimage and a right image and may be selected from a larger source image.There are systems disclosed, as generally discussed in U.S. Pat. No.4,634, 384 and U.S. Patent Publication No. 2003/0067476, both of whichare herein incorporated by reference in their entirety, whichdifferentially vary two-dimensional image properties based on knowledgeof a foveated region. These systems do not address binocular conditions.

There are systems disclosed, as generally discussed in U.S. PatentPublication No. 2012/0200676 that use eye tracking to adjust a computergraphics model in a scene so that a stereoscopic image rendered from themodel is altered. These systems adjust the entire model and do notresult in differential changes across the stereoscopic image in order tovary depth mapping from scene to image inside and outside the region ofbinocular interest.

Additionally, there are systems disclosed, as generally discussed inU.S. Pat. No. 6,198,484, which are herein incorporated by reference intheir entirety, that alter the stereoscopic image presentation based onhead position and/or eye tracking to account for motion parallax. Thesesystems do not make differential changes across the stereoscopic imagein order to vary depth mapping from scene to image inside and outsidethe region of binocular interest.

FIG. 1 illustrates a binocular imaging system which may include abinocular image display 5 for presenting different images perceptuallyand substantially simultaneously to the left and right eyes. The viewerwho sees the binocular images presented by image display 5 in which lefteye images are seen by the left eye 1 with a field of view of thedisplay 3, and right eye images are seen by the right eye 2 with a fieldof view of the display 4. The binocular imaging system may also includea gaze tracking element 6 that may identify one or both gaze directionsof the left eye and right eye 7, 8, respectively, and may include a wayto calculate the viewer's binocular region of fixation 9. The gazetracking element 6 may calculate the viewer's binocular region offixation 9 by way of any appropriate processing or computing system thatmay include a computer readable medium. The binocular region of fixation9 may be a three dimensional region in which the location varies withthe direction of gaze. The binocular imaging system may also include away to alter the displayed images in such a way as to affect the localimage depth content both in and surrounding the binocular region offixation.

In one embodiment a method may discuss a fixated region which hassubstantially zero disparity. This embodiment may provide a method inthe image controller 10 that may alter the subsequently displayed imagesin response to the change of the viewer's region of binocular fixation 9between the currently displayed binocular image and the subsequentlydisplayed binocular image, as generally illustrated in FIGS. 6, 2, and1.

In FIG. 6, step S60 forms the currently displayed images as a left image22 and right image 23. The currently displayed images may be selectedfrom larger source images 20 and 21, as shown in FIG. 2. In step S61 ofFIG. 6, the current binocular images are displayed by the image display5 of FIG. 1.

The left and right images 22 and 23 of FIG. 2 may contain images ofobjects for example 24, 25, 26 whose horizontal location may differ.This horizontal difference between images of the same object indifferent locations in left and right eye views is known as imagedisparity and its magnitude and sign controls the depth perceived by theviewer when they binocularly fuse the left and right image.

Also in FIG. 6, as shown in step S62, the image controller 10 of FIG. 1,receives input from the gaze tracker 6 and uses this input to calculatethe subsequent binocular region of fixation.

Continuing the discussion with respect to FIGS. 1, 2, and 6, in step S63the controller determines any change in binocular fixation between thecurrent and subsequent fixations, if there is no change in the binocularfixation between the current and subsequent fixations, the controllercontinues at step S62.

In step S64, when there is a change in binocular fixation, the imagecontroller 10 calculates which subsequent object in the scene is beingimaged where it is the closest object to the viewer in the region ofbinocular fixation. In step S65 the controller compares the imagedisparity of the subsequent object with zero, if the disparity is zero,the controller continues at step S62.

In step S66 the controller uses the image disparity of the subsequentobject to adjust the subsequently displayed images so that the imagedisparity of the subsequent object becomes substantially zero. Onemethod is illustrated in FIGS. 3, 4 and 5.

FIG. 3 illustrates the currently displayed binocular image pair 22, 23where the region of binocular fixation is aligned with the object 25 inthe image. FIG. 3 also includes an illustrative line showing an objectwith zero disparity in the displayed image. The horizontal disparitybetween the left and right images for object 25 is zero, as is indicatedby the illustrative line 30.

In FIG. 4, the viewer's region of binocular fixation has moved fromobject 25 to object 24 and the image controller 10 reacts by creatingthe subsequently displayed images 40 and 41. Region 40 illustrates anewly selected region to be displayed in the left eye view and region 41illustrates a newly selected region to be displayed in the right eyeview. The image controller achieves this by finding the disparity forobject 24 and, in this case, then slides the right image window to theright by that number of pixels so the disparity of subsequently fixatedobject 24 is zero.

The resulting subsequently displayed images are shown in FIG. 5 wherethe horizontal disparity for object 24 is now zero as shown by theillustrative line 50. FIG. 5 includes an illustrative line to show thenew object with zero disparity in the displayed image.

In step S66 of FIG. 6, the subsequently displayed images 40 and 41 arenow made the currently displayed images and control returns to step S61where the currently displayed images are displayed by the image display5.

Another related embodiment may include a fixated region which may havesubstantially zero disparity. This embodiment adjusts the imagery in asimilar manner to that of the previous embodiment but uses the gazetracker to determine the disparity within the fixated region. When theleft and right eye images are displayed on a stereoscopic device, thegaze detector can determine the plane of fixation from the differencebetween left and right eye screen fixation points. If the plane offixation is in front of the display screen, for example when the lefteye's fixation point on the display is to the right of the right eye'sfixation point, it can be inferred with little to no calculation thatthe imagery in the fixated region has negative disparity. Shifting theimagery relative to each other can remove this negative disparity andprovide for substantially zero disparity as in the previous embodiment.

Yet another embodiment may have a fixated region with disparity and thesurrounding region may have substantially no disparity. This embodimentmay provide a method in the image controller 10 that may alter thesubsequently displayed images in response to the change in the viewer'sregion of binocular fixation 9 between the currently displayed binocularimage and the subsequently displayed binocular image. Discussion isprovided in FIGS. 11, 7, 8, 9, and 10.

In FIG. 11, step S110 forms the currently displayed images as in FIG. 7for left eye 70 and right eye 71, respectively. To form the currentlydisplayed images, a 3D region of fixation is measured or estimated andis projected into the image plane 72 to form the binocular region offixation. Any image information outside this region 72 in the images 70and 71 is the same in each image. This may be a monocular imageproviding the same information to both the viewers' eyes. Any imageinformation inside region 72 is binocular. This image information may berendered or captured or synthesized with binocular disparityinformation. The result is illustrated in FIG. 8. The objects 75 and 73included in the scene, may be outside region 72 and may have nobinocular disparity shown by illustrative lines 80 and 81 in FIG. 8.Meanwhile, the object 74 inside region 72 has binocular disparity 84 andis shown by illustrative lines 82 and 83.

In step S111 of FIG. 11, the current binocular image pair 70 and 71 isdisplayed on the image display 5 of FIG. 1. In step S112 the imagecontroller 10 of FIG. 1 receives input from the gaze tracker andcalculates the subsequent region of binocular fixation 92 in FIG. 9. Instep 113 if the region of binocular fixation has not changed, thencontrol returns to step S112 of FIG. 11.

In step S114 of FIG. 11, the region of binocular fixation has changedand the image controller calculates the depth range of the subsequentregion of binocular fixation in the scene. Continuing the discussion, instep S115 the depth range information may be used to create a formedbinocular image for the region of binocular fixation 92 which may becombined with a monocular image to form the subsequently displayedimages 90 and 91 as shown in FIG. 9. The result is highlighted in FIG.10, in which the region with binocular disparity 92 includes object 73which has binocular disparity 104. Additionally, objects 74 and 75outside the region of binocular fixation no longer have any disparity asillustrated by lines 100 and 101.

Finally, in step S116 the currently displayed binocular image becomesthe subsequently displayed binocular image and control returns to stepS111.

FIG. 12 is a schematic diagram illustrating one embodiment of a gazetracking system. FIG. 12 provides an example of a viewer and a display153 and the different elements in display space. In FIG. 12, a viewer'seyes 155 are looking at a displayed binocular image. The left eye of theviewer may be looking in a direction referred to as a left eye gazedirection 120 and the right eye of the view may be looking in adirection referred to as a right eye gaze direction 121. The viewer'seyes 155 may be tracked by a gaze tracking system 6. As illustrated inFIG. 12, scene 122 depicts a scene as perceived in a fused binocularimage and region 160 RBF_(d) may be a region of binocular fixation indisplay space. The gaze tracking system 6 may provide gaze trackinginformation which may be used to calculate a viewer's subsequent regionof binocular fixation, among other things.

FIG. 13 is a schematic diagram illustrating one embodiment of a scenespace. FIG. 13 provides an example of cameras and a scene space. In FIG.13, cameras 154 may be located in a position for capturing images of ascene. Located by the cameras 154 may be a depth measurement system 156.Although the depth measurement system 156 is illustrated as centrallylocated between the cameras 154, this is for discussion purposes onlyand not of limitation as the depth measurement system may be located inother positions with respect to the scene space as appropriate. In FIG.13, the range 150 may be the total scene depth range and the scene depthrange 163 may represent the scene depth range to map to a depth budgetin display space. Additionally, as depicted in FIG. 13, the region 162may be the region of binocular interest, RBI, 162 and the region 161 maybe the region of binocular fixation projected into scene space, RBF_(s)161.

In yet another embodiment, depth mapping from scene to display space andmay be determined by the region of binocular fixation in the displayspace. This embodiment is described referring to FIGS. 20, 15, 16, 17,18, and 19.

Referring to FIG. 15, the viewer is looking at a first current binocularimage and sees perceived depth in it, in this case, within somepre-determined perceived depth budget 151. The specific mapping of depthfrom the scene space being imaged 150 to the display space perceiveddepth budget 151 can be calculated using a pre-existing camera controlalgorithm such as in reference U.S. Pat. No. 6,798,406, given a depthmeasurement element 156 to determine the range of depth in the scene.The depth range 150 in the scene can, for example, be computed from adepth map in synthetic scenes or an optical or laser range finder inreal scenes.

Referring to the flowchart in FIG. 20, a first current binocular imageis produced in step S200 and then displayed in step S201. Referring toFIG. 16, while the viewer is looking at the displayed binocular imagetheir gaze is being tracked using a gaze tracking element 6 of FIG. 1and this information is used to determine the region of binocularfixation in display space RBF_(d) 160.

The RBF_(d) is used by the image controller 10 of FIG. 1 to calculatethe equivalent region of binocular fixation in the scene space RBF_(s)161. RBF_(s) 161 may then be used to calculate the region of binocularinterest in scene space RBI 162. RBI encompasses any objects that fallin a volume of space that is a super-set of the RBF_(s). The RBI may beany convenient three-dimensional shape including, but not limited to, aparallelepiped, cylinder, ellipse, frustum, and so forth.

In step S204 of FIG. 20, once the RBI is calculated, the scene depthrange that is to be mapped to the perceived depth budget can be found bycalculating the depth extent of the RBI, illustrated in FIG. 16 as 163.This allows the application of any depth mapping camera controlalgorithm as generally discussed in U.S. Pat. No. 6,798,406 to generatea subsequent binocular image in step S205 and set this for display instep S206.

FIGS. 17, 18 and 19 illustrate the image controller's response to a realtime change in the viewer's region of binocular fixation RBF_(d).

In FIG. 17 the RBF_(d) has changed to a different position in thedisplay space 170 as detected by the gaze-tracking element 6 andcalculated by the image controller 10. The image controller 10 thencalculates a new RBF_(S) 180 as illustrated in FIG. 18 and additionallycalculates a new RBI 181 that forms a volume of space that is a supersetof the RBF_(S) 180. Depending on the contents of the scene the new RBImay be larger, or smaller than that the current value. The scene depthrange 182 to be mapped to the depth budget 151 will then also change.Once the scene depth range 182 is known, the application of any depthmapping camera control algorithm as generally discussed in U.S. Pat. No.6,798,406, can map the newly calculated scene depth range 182 to thedisplay perceived depth budget 151.

The result in FIG. 19 shows the new mapping of scene depth to depthbudget. The technical benefit is that as the viewer's gaze moves aroundthe scene, as displayed in the binocular image, the depth in the RBF_(d)and corresponding RBI is continuously optimized to fit the availabledepth budget 151.

Of importance is that this embodiment will also operate in animatedscenes where the RBI changes due to scene changes even when the RBF_(d)region of binocular fixation does not change. This is measured by adepth measure element 156, which in computer graphics may be a depthbuffer, or in photography, may be a range finder such as an optical orlaser device.

Yet another embodiment may include a fixated region which may havepreferred disparity and the surrounding region may have a differentdisparity where the total disparity does not exceed a predeterminedlimit using variable z-region mapping. This embodiment may provide amethod in the image controller 10 that is able to alter the subsequentlydisplayed images in response to the change in the viewer's region ofbinocular fixation 9, between the currently displayed binocular imageand the subsequently displayed binocular image, with reference to FIGS.26, 21, 22, 23, 24, and 25.

Referring to the flowcharts in FIG. 26 and FIG. 21. In step S260 a firstbinocular image is formed. This can be formed when a scene depth range150, as shown in FIG. 15, is mapped to a perceived depth range 151 usinga method as disclosed in references such as U.S. Pat. No. 6,798,406,U.S. Patent Application Pub. No. US 2011/7,983,477 or U.S. Pat. No.8,300,089 both of which are herein incorporated by reference in theirentirety. The first current binocular image is then displayed in stepS261.

Referring to FIG. 22, in step S262 the image controller receives inputfrom the gaze tracker 6 this allows identification of the region ofbinocular fixation RBF_(d) 160 in display space. From this, the regionof binocular fixation in scene space RBF_(s) 161 can be found and withadditional input from the scene depth measurement element 156 the regionof binocular interest RBI 162 in the scene can be calculated. Knowingthe RBI or scene depth range 150, it is possible to calculate 163, whichis the scene depth range 150 to be mapped to the perceived depth budget151 in display space. In this instance 163 is approximately the same asthe scene depth range 150, for example, the RBI has not changed, and sono change in the depth mapping is required and step S263 can return tostep S262.

Alternatively referring to FIG. 23, the viewer's gaze has changed andthe input from the gaze tracker identifies a subsequent RBF_(d) 230.Then as illustrated in FIG. 24 this allows a subsequent RBF_(s) 240 tobe calculated and from this the subsequent RBI 241. As the subsequentRBI 241 is now different from the current RBI 162 (as illustrated inFIG. 16), execution continues at step S264 and the subsequent scenedepth range 163 is calculated.

Step S265 then calculates a new mapping of depth from the scene to thedisplay space. FIG. 25 illustrates one way to implement the mapping forstep S265 using a multi-region depth mapping algorithm such as generallydisclosed in U.S. Pat. No. 7,983,477. Here the RBI can be considered asa region of interest 211 dividing the scene into three regions includinga nearer region 210 and a further region 212. These are then mapped tothree corresponding regions in the scene space, 213, 214, and 215.Because the regions 213, 214, and 215 may differ in the amount ofperceived depth allocated to them, the region of interest 211 and hencethe RBI can be given a preferential amount of scene depth compared tothe near and far regions. Additionally it prevents any objects of thescene from appearing outside of the perceived depth range 151, such asfor example, the single region mapping as illustrated in FIG. 19. Oncethe subsequent image is formed it is set to be the current image in stepS266 and control return to step S261.

Yet another embodiment may include a fixated region which may havepreferred disparity and the surrounding region may have a differentdisparity where the total disparity does not exceed a predeterminedlimit using variable camera parameters in one or two dimensions. Thisembodiment may provide a method in the image controller 10 that mayalter the subsequently displayed images in response to changes in theviewer's region of binocular fixation 9 between the currently displayedbinocular image and the subsequently displayed binocular image, withreference to FIGS. 27, 28, 29, and 30.

In FIG. 14, the flowchart step S300 forms a first binocular image. Thisimage may be formed when a scene depth range 150 is mapped to aperceived depth range 151 using a method as disclosed in references suchas U.S. Pat. Nos. 6,798,406, 7,983,477, or 8,300,089. The currentbinocular image 5 is displayed in step S301.

Referring to FIG. 14, in step S302, the image controller receives inputfrom the gaze tracker 6 and this may allow identification of the regionof binocular fixation RBF_(d) 160 in display space. From this, theregion of binocular fixation RBF_(s) 161 in scene space can be found andwith additional input from the scene depth measurement element 156 theregion of binocular interest RBI 162 in the scene can be calculated.Knowing the RBI it is possible to calculate 163 the scene depth range tobe mapped to the perceived depth range 151 in display space. If the RBIhas not changed no change in the depth mapping is required and step S303returns to S302.

Referring to FIG. 27, when an RBI has been identified, a locally varyingdepth mapping from scene space to display space can be calculated inS304. This can vary the stereoscopic camera parameters used to capturethe image. For example, a full stereoscopic 3D effect near the RBI maychange to a simple 2D effect outside the RBI, as illustrated in FIG. 27.

If the RBI changes, as shown in FIG. 28, then the region of the imagewith full stereoscopic 3D effect can be changed too. The benefit becomesany region of the displayed image away from the RBI may be rendered froma single camera viewpoint, saving the computational costs of renderingtwo images while keeping the foveated region at the highest possiblequality.

FIG. 29 illustrates how the camera parameters used for rendering in stepS305 can vary in one dimension depending where in the stereoscopicimage, different scene elements may appear. In this case elements awayfrom the RBI may be rendered from a single central camera viewpoint C,while elements in the RBI are rendered with a stereoscopic camerasetting A₀, which may be calculated using methods as generally discussedin U.S. Pat. No. 6,798,406. In the zone between the full stereoscopicand the two-dimensional image regions the camera setup is linearlyinterpolated, with the interaxial separation A₁ reducing until theindividual use of the single central camera C is appropriate.

A further embodiment of this approach is to vary the camera parameterswith vertical as well as horizontal element position, so that theregions of the image that are horizontally and vertically close to theRBI, are rendered with full stereoscopic effect.

One possible implementation of these embodiments is illustrated inListing 1 which provides an outline of a GLSL, as generally discussed inOpenGL Reference Pages, at http://www.opengl.org/documentation/glsl/, avertex shader solution for interpolating the camera parametersappropriate for projecting and shading vertices in a real time computergraphics system, in order to produce a foveated stereoscopic renderingeffect.

Listing 1: 1 // Listing 1, Method for 1D and 2D foveated stereoscopiccameras using GLSL. 2 // This method is called once to draw left pictureand once for right picture. 3 4 in vec4 inPosition, inNormal; // Inputinformation about each vertex. 5 out vec4 vsColor; // Output colour forper-vertex shading. 6 7 uniform mat4 modelMatrix; // Modeltransformation is common to all views. 8 9 uniform mat4 viewMatrix,projectionMatrix; // Left or right stereoscopic camera position. 10uniform mat3 normalMatrix; 11 12 uniform mat4 cViewMatrix,cProjectionMatrix; // Centre view camera position. 13 uniform mat3cNormalMatrix; 14 15 uniform float rABound; // Scene space boundary tostart cross fading. 16 17 // The scene space origin of the foveatedregion. 18 uniform float originX, originY; 19 20 uniform vec3lightDirection; // Used to calculate shaded colour at the vertex. 21uniform vec4 lightColor, ambientColor; 22 23 void main(void) 24 { 25  vec3 normal, normLightDir; 26 27   float vertX, vertY; 28   vec4vmPosition ; 29   float fadeZone = 30.0; // Width of cross fade region.30 31   float weight, invWeight, weighty, invWeightY ; 32 33   mat4MVPMat; 34   mat4 cMVPMat; 35 36   mat4 weightedMVPMat; 37   mat3weightedNormMat; 38 39   vmPosition = (cViewMatrix * modelMatrix) *inPosition; 40   vertX = abs( vmPosition.x + originX ); 41   vertY =abs( vmPosition.y + originY ); 42 43   weight = (vertX − rABound) /fadeZone ; 44   weightY = (vertY − rABound) / fadeZone ; 45 46   weight= max( weight, 0.0 ) ; // Calculate weight in X direction. 47   weight =min( weight, 1.0 ) ; 48   weightY = max( weightY, 0.0 ) ; // Calculateweight in the Y direction. 49   weightY = min( weightY, 1.0 ) ; // 50  weight = max( weight, weightY ); // Choose to use max of X and Yweights. 51   invWeight = 1.0 − weight; // NB 1.0 == (weight + invWeight) 52 53   // Calculate weighted transformation matrix for the surfacenormal. 54   weightedNormMat = (invWeight * normalMatrix) + (weight *rANormalMatrix) ; 55   normal = weightedNormMat * inNormal; 56   normal= normalize( normal ); 57   normLightDir = normalize( lightDirection );58 59   // Output vertex colour using weighted normal projection and adiffuse lighting model. 60   vsColor = ambientColor * 0.3 + lightColor *max(dot(normal, normLightDir), 0.0); 61 62   // Calculate weightedprojection matrix for the vertex geometry. 63   MVPMat =projectionMatrix * viewMatrix * modelMatrix ; 64   cMVPMat =cProjectionMatrix * cViewMatrix * modelMatrix ; 65   weightedMVPMat =(invWeight * MVPMat) + (weight * cMVPMat) ; 66 67   // Output theprojected vertex position using the weighted MVP matrix. 68  gl_Position = weightedMVPMat * inPosition; 69 }

The shader described in Listing 1 is called once for the left eye viewand once for the right eye view. Lines 4 through 21 declare thevariables that are set before the shader runs. Additionally, lines 9 and10 describe the camera parameters needed for a left or right eyeposition. Lines 12 and 13 describe the camera parameters for a singlecentral monoscopic view. Also, lines 25-37 declare the variables thatmay be used during the calculations of the foveated camera parameters.

Of note are the variables rABound on line 15 and the fadeZone on line29, which in combination with the foveated region origin given byoriginX and originY on line 18, primarily determine the position andextent of the foveated region where stereoscopic rendering will beimplemented. At the boundary of this region given by rABound the cameraparameters will be interpolated over a scene distance primarilydetermined by fadeZone to become a monoscopic image.

The appropriate weighting to do this is calculated between lines 39 and51. Note, if the weight value calculated has a value of 1.0 then themonoscopic zone has been reached and the calculations between lines 53and 68 may be calculated once for the left camera and not the rightcamera view. Resulting in substantial computational savings compared tocalculating the projection and shading calculations separately for botheyes.

Where these calculations are appropriate, for the foveated stereoscopiczone and for one camera's view in the monoscopic zone then:

-   -   a. Lines 53-60 describe how the surface normal vectors are        transformed using the weighted normal transformation matrix and        then used to calculate a shaded color value for the vertex using        for illustration a single light Lambertian shading model.    -   b. Lines 62-68 describe how the vertex position is transformed        using the weighted model-view-projection matrix.

The resulting shaded color vsColor and the transformed vertex positiongl_Position are passed onto the next stage in the computer graphicsrendering pipeline.

Yet another embodiment may include a fixated region which may havepreferred disparity and the surrounding region may have a differentdisparity in which the total disparity does not exceed a predeterminedlimit using variable camera parameters in three dimensions. Using atri-linear interpolation model to calculate the weight value will allowthe depth dimension to be foveated as well as the two image dimensions.This can be implemented using a camera model as described in U.S. Pat.Nos. 7,983,477 or 8,300,089 in which the mapping of the depth dimensionis variable.

The benefit may include optimizing the depth presentation of the imageseen in the foveated region while reducing the computational or depthbudget demands for drawing the image regions representing the scenein-front and behind this region in depth. For example, in a driving gamethe best image quality is given to the region of the scene to which thedriver is attending.

Another embodiment may have a fixated region which may have preferreddisparity and the surrounding region may have a different disparity inwhich the total disparity does not exceed a predetermined limit by anyof the above methods and multiple fixated regions are computed to allowmultiple gaze-tracked viewers to watch the same screen and attend todifferent parts of the screen. For example, if there are multipleviewers of the same screen then the multiple viewers are generalunlikely to be fixating on the same region of the image. This can besolved with multiple gaze tracking devices, for example each wearing ahead mounted Eye Link II eye tracker from SR Research Ltd., Mississauga,Ontario, Canada. Using the eye tracking information, each viewers' RBIcan be calculated and used to determine the individual weights used tocontrol the camera parameters across the screen.

Continuing this discussion, this embodiment enables multiple viewers tolook at a gaze tracked image and although temporally varying the regionsof interest are often similar enough between viewers, as generallydisclosed in Active Vision, Findlay and Gilchrist, OUP, 2003, this mayresult in savings in image regions to which none of the viewers mayattend.

Further, in yet another embodiment, a fixated region may have preferreddisparity and the surrounding region may have a different disparity inwhich the total disparity does not exceed a predetermined limit by anyof the above methods and the disparity is temporally modified todistinguish the region or a part of one of the regions. In the exampleutilizing temporal disparity modification a viewer relocates theirfixation to a region of interest and then after a period the disparityin the region of fixation is altered to introduce a noticeable depthalteration. In turn this introduces a detectable change in theconvergence and/or divergence of each eye allowing the eyes meanfixation point to be calculated.

Continuing this discussion, this can be used to provide an embodimentfor user interfaces in which the icons on a desktop are givendifferential depth within a finite but coarse fixation region. One iconmay then be temporally modified. If differential changes in convergenceand/or divergence of the eyes is detected it can then be inferred thatthe eyes are fixating the varying icon.

When the eyes differential change in convergence is detected the iconcould then be primed for selection. Once an icon is primed in this waythe viewer can activate the primed icon by pressing a button, longfixation, blinking, any combination thereof, and so forth.

A further enhancement can allow the system to dynamically adapt to theuser, if the system does not detect any change in eye convergence and/ordivergence it can alter which icon is varied in disparity and eventuallyprime a specific icon when it eventually detects a temporal change ineye convergence.

The benefit of temporally altering the disparity here is to use theinduced temporal changes in eye convergence and/or divergence toincrease the overall system confidence in regards to which icon is beingattended too.

Further, temporal disparity modification might not be continuous. Analternative use of the temporal variation in disparity may be to attractattention to regions of a binocular image. In this case regions outsidethe region of attention can be varied to attract the viewer to attend tothem, for example because of the looming effect as generally discussedin Basic Vision, Snowden, Thompson, Troscianko, OUP, 2006.

A direct benefit of this is in warning systems in which there is a needfor the viewer's attention to be drawn to urgent information in a regionoutside their current region of fixation. In this case, the disparity ofa warning icon outside the region of interest is varied temporally. Oneexample of such an icon may be a low battery warning indicator. Althoughit is unlikely that it would be in the viewer's region of fixation, itis important to draw the viewer's attention to the icon when the batteryis lower than a predetermined capacity remaining. It may be evident tothose skilled in the art that there are many other icons in which thismay benefit in many types of information presentation systems

Yet another embodiment may include a fixated region which may havepreferred disparity and the surrounding region may have a differentdisparity in which the total disparity does not exceed a predeterminedlimit by any of the above methods and one or more of the following imagequality parameters are also varied. The varied image quality parametersare listed below.

The first image quality parameter that may be varied is color quality interms of bits per pixel, or other color representation scheme. This maybenefit the highlighting of certain areas using enhanced or reducedcolor representations and the benefit of reduced color computation timeand/or energy saving in the GPU and/or reduced display bandwidth inthose areas of reduced color representations.

Another image quality parameter that may be varied is grey level qualityin terms of bits per pixel, or other grey level representation scheme.This may provide the benefit of highlighting certain areas usingenhanced or reduced grey level representations and the benefit ofreduced grey level computation time and/or reduced display bandwidth inthose areas of reduced grey level representations.

Another image quality parameter that may be varied is image luminance interms of total light power, for example, by using less of the displaybrightness range, or by using a high dynamic range display with anability to boost brightness in a particular region of an image. This hasbenefits including, but not limited to, reduced power usage in regionsof the screen with lower brightness and lower visibility of highfrequency image artifacts such as aliasing in the lower brightnessregions of the image when lower resolution image content is used.

Another image quality parameter that may be varied is image contrast,for example, by changing the gamma curve of the displayed image. Thishas the benefit of masking the visibility of other performance changes.For example, reduced resolution can result in blockiness in the imagewhich can be masked with a low pass filter.

Another image quality parameter that may be varied is image spatialfrequency content, for example, using high, low or band pass filters. Inone example, regions can be blurred to reduce computation and reducespatial resolution that may be appropriate in some regions of the image.This may contribute to reducing computational demands in regions of thescreen with lower spatial frequency.

Another image quality parameter that may be varied is image temporalfrequency using higher or lower image refresh rates in different areasof the screen. This may contribute to reducing computational and displaybandwidth conditions in regions of the screen with lower temporalfrequency.

Another image quality parameter that may be varied is scene geometrycontent in which the quality of the computer graphics model is varied bychanging the quality of geometric model used to represent objects. Thismay contribute to reducing computational bandwidth conditions in regionsof the screen with reduced quality geometric models, for example, lowernumber of triangles in geometry meshes.

Another image quality parameter that may be varied is scene textureimage content in which the quality of the computer graphics modeltexture images is varied. This may contribute to reducing computationalbandwidth conditions in regions of the screen with reduced qualitytexture images, for example lower resolution images.

Another image quality parameter that may be varied is computer graphicsrendering parameters so that effects including specular highlights,reflection, refraction, transparency vary in quality between the imageregions. This may contribute to reducing computational bandwidthconditions in regions of the screen with reduced graphics effects.

Another image quality parameter that may be varied is disparity gradientin terms of maximum gradient allowed in one region compared to anotherregion. This may contribute to improving perceived image quality inimage regions in which disparity gradient may otherwise be too high tofuse the images comfortably, or so high that it may be detrimental totask performance.

As discussed herein and in at least one embodiment, binocular fixationmay be a volume in space around the point of intersection of the twooptical axes of the eyes.

As discussed herein and in one at least embodiment, binocular image maybe a pattern of light that generates separate stimulus for the two eyes.This may include multiple resolvable views in different directions overeach pupil. It can, for example, be generated using discrete views orcontinuous wave fronts, technically produced using stereoscopic,auto-stereoscopic, multiscopic or holographic optical devices.

As discussed herein and in at least one embodiment, a binocularly fusedimage may be a perceptually single view (cyclopean view) of the worldformed by fusing two images. This may provide a sensation of (perceived)depth in the scene.

As discussed herein and in at least one embodiment, capture may be aprocess that generates a binocular image from the real world orsynthetic data. The binocular image may be using optical functions suchas still or motion cameras, or rendered using computer graphics or otherimage synthesis mechanisms.

As discussed herein and in at least one embodiment, depth budget may bea range of perceived depth, implying a range of binocular disparity thathas been chosen as the total limit of perceived depth seen in abinocularly fused image. The depth budget may be chosen for comfort ortechnical reasons.

As discussed herein and in at least one embodiment, depth mapping may bethe process of capturing depth from a scene and reproducing it asperceived depth in a binocular image.

As discussed herein and in at least one embodiment, depth measurement ordepth measurement element may be a mechanism, real or virtual, formeasuring distance, depth, of a surface from a fixed point. In the realworld scenes this may be a laser rangefinder, an optical range finder,and so forth. In synthetic scenes this may be a depth map, or ageometric calculation that measures the distance from a fixed point. Inmost or all cases the depth measurements may be relative to cameraposition and may be used to calculate a depth mapping from scene spaceto the perceived image space.

As discussed herein and in at least one embodiment, gaze tracking mayinclude methods for following the eyes movements to determine thedirection of gaze. These can be implemented with devices that employdirect contact with the eye or are remote measurement elements that, forexample, follow reflections of light from the eye.

As discussed herein and in at least one embodiment, foveated images maybe an image that is perceived in the foveal region of the retina.

As discussed herein and in at least one embodiment, a foveated regionmay be a region is an image or a scene that is perceived in the fovealregion of the retina.

As discussed herein and in at least one embodiment, an image may be apattern of light that can be detected by the retina.

As discussed herein and in at least one embodiment, disparity may be adifference in the location of a point, normally horizontal, in whichhorizontal is taken to be defined by the line joining the two eyes andthe disparity is measured on the retina.

As discussed herein and in at least one embodiment, a monoscopic imagemay be an image that is substantially the same when viewed from anydirection. If presented to both eyes, both eyes receive substantiallythe same pattern of light. For example, a standard 2D TV presents amonoscopic stimulus, each pixel broadcasts the substantially similar orthe same light in all viewing directions.

As discussed herein and in at least one embodiment, a region ofbinocular fixation in display space may be RBF_(d) or a volume indisplay space that corresponds to the region of overlap of the gazezones of the two eyes.

As discussed herein and in at least one embodiment, a region ofbinocular fixation in scene space may be RBF_(s) or a volume in scenespace that corresponds to the region of overlap of the gaze zones of thetwo eyes.

As discussed herein and in at least one embodiment, a region ofbinocular interest may be an RBI or a volume of scene space thatincludes the region of binocular fixation and is extended to include thescene limited by the gaze zones of the two eyes.

As discussed herein and in at least one embodiment, scene depth rangemay be a range of depth measured in the scene, usually that may bemapped to a range of perceived depth in a fused binocular image.

As discussed herein and in at least one embodiment, a stereoscopic imagemay be an image that includes a pair of images that are presentedseparately to each eye. The implication is that the position of each ofthe viewer's eyes is important when viewing a stereoscopic image as adifferent pattern of light is received on the two retinas.

As discussed herein and in at least one embodiment, rendering may be theprocess of creating an image from a synthetic scene.

As discussed herein and in at least one embodiment, synthetic scenes maybe scenes in a computer graphics, virtual world or depth-based imagethat may be physically real, though my represent physically real scenes.

As discussed herein and in at least one embodiment, a view may be aunique image visible in a single direction.

As discussed herein and in at least one embodiment, a scene may be areal world or synthetic scene which is being captured and thenreproduced as a binocular image.

As may be used herein, the terms “substantially” and “approximately”provide an industry-accepted tolerance for its corresponding term and/orrelativity between items. Such an industry-accepted tolerance rangesfrom less than one percent to ten percent and corresponds to, but is notlimited to, component values, angles, et cetera. Such relativity betweenitems ranges between less than one percent to ten percent.

It should be noted that embodiments of the present disclosure may beused in a variety of optical systems. The embodiment may include or workwith a variety of projectors, projection systems, optical components,computer systems, processors, self-contained projector systems, visualand/or audiovisual systems and electrical and/or optical devices.Aspects of the present disclosure may be used with practically anyapparatus related to optical and electrical devices, optical systems,display systems, presentation systems or any apparatus that may containany type of optical system. Accordingly, embodiments of the presentdisclosure may be employed in optical systems, devices used in visualand/or optical presentations, visual peripherals and so on and in anumber of computing environments including the Internet, intranets,local area networks, wide area networks and so on.

Regarding the disclosed embodiments in detail, it should be understoodthat the embodiment is not limited in its application or creation to thedetails of the particular arrangements shown, because the embodiment iscapable of other arrangements. Moreover, aspects of the embodiment maybe set forth in different combinations and arrangements to defineembodiments unique in their own right. Also, the terminology used hereinis for the purpose of description and not of limitation.

While various embodiments in accordance with the principles disclosedherein have been described above, it should be understood that they havebeen presented by way of example only, and not limitation. Thus, thebreadth and scope of this disclosure should not be limited by any of theabove-described exemplary embodiments, but should be defined only inaccordance with any claims and their equivalents issuing from thisdisclosure. Furthermore, the above advantages and features are providedin described embodiments, but shall not limit the application of suchissued claims to processes and structures accomplishing any or all ofthe above advantages.

Additionally, the section headings herein are provided for consistencywith the suggestions under 37 CFR 1.77 or otherwise to provideorganizational cues. These headings shall not limit or characterize theinvention(s) set out in any claims that may issue from this disclosure.Specifically and by way of example, although the headings refer to a“Technical Field,” the claims should not be limited by the languagechosen under this heading to describe the so-called field. Further, adescription of a technology in the “Background” is not to be construedas an admission that certain technology is prior art to anyembodiment(s) in this disclosure. Neither is the “Summary” to beconsidered as a characterization of the embodiment(s) set forth inissued claims. Furthermore, any reference in this disclosure to“invention” in the singular should not be used to argue that there isonly a single point of novelty in this disclosure. Multiple embodimentsmay be set forth according to the limitations of the multiple claimsissuing from this disclosure, and such claims accordingly define theembodiment(s), and their equivalents, that are protected thereby. In allinstances, the scope of such claims shall be considered on their ownmerits in light of this disclosure, but should not be constrained by theheadings set forth herein.

What is claimed is:
 1. A method for varying binocular image content,comprising: displaying a current binocular image comprising left andright eye views, wherein the left and right eye views each comprise afirst portion of respective left and right eye source images; usinginputs from the current binocular image and a gaze tracker to calculatea subsequent region of binocular fixation; determining any change inbinocular fixation between a current region of binocular fixation andthe subsequent region of binocular fixation; calculating a disparityrange of the subsequent region of binocular fixation when there is achange in binocular fixation between the current region of binocularfixation and the subsequent region of binocular fixation; determiningwhether the disparity range is substantially zero; choosing new left andright eye views to create a subsequently displayed binocular image whenthe disparity range is not substantially zero, wherein a nearest objectin the subsequent region of binocular fixation has a disparity ofsubstantially zero in the subsequently displayed binocular image, andwherein the new left and right eye views each comprise a second portionof the respective left and right eye source images, wherein the firstand second portions of the left eye source image are different portions,and the first and second portions of the right eye source image aredifferent portions; and making the currently displayed image, thesubsequently displayed binocular image.
 2. The method for varyingbinocular image content of claim 1, further comprising receiving asecond input from the gaze tracker and using the second input from thecurrent binocular image and the gaze tracker to re-calculate thesubsequent region of binocular fixation when the subsequent region ofbinocular fixation has not substantially changed, and if it isdetermined that the subsequent region of binocular fixation, afterre-calculation, has changed, then calculating the disparity range of thesubsequent region of binocular fixation.
 3. The method for varyingbinocular image content of claim 1, further comprising receiving a thirdinput from the gaze tracker and using the third input from the currentbinocular image and the gaze tracker to further re-calculate thesubsequent region of binocular fixation when the disparity range isapproximately zero, and if it is determined that the subsequent regionof binocular fixation, after further re-calculation, has changed, thenre-calculating the disparity range of the subsequent region of binocularfixation.
 4. The method for varying binocular image content of claim 1,further comprising determining, with the gaze tracker, the disparitywithin the fixated region, wherein the gaze tracker determines the planeof fixation from the difference between left eye and right eye screenfixation points.
 5. The method for varying binocular image content ofclaim 1, wherein determining whether the disparity range issubstantially zero further comprises comparing the image disparity ofthe subsequent object with zero, wherein the subsequent object is beingimaged where it is the closest object to a viewer in the region ofbinocular fixation.
 6. The method for varying binocular image content ofclaim 1, further comprising altering a subsequently displayed image inresponse to the change in the region of binocular fixation between thecurrently displayed binocular image and the subsequently displayedbinocular image.
 7. The method for varying binocular image content ofclaim 1, further comprising forming a currently displayed binocularimage.
 8. The method for varying binocular image content of claim 1,wherein forming a currently displayed binocular image further comprisesestimating a 3D region of fixation and projecting the 3D region offixation into an image plane to form a binocular region of fixation. 9.The method for varying binocular image content of claim 8, wherein thecurrently displayed binocular image is formed as a left image and aright image.
 10. The method for varying binocular image content of claim9, wherein the currently displayed binocular image is selected from alarger source image.
 11. The method for varying binocular image contentof claim 1, further comprising receiving a second input from the gazetracker and using the second input from the current binocular image andthe gaze tracker to re-calculate the subsequent region of binocularfixation when the disparity range is approximately zero, and if it isdetermined that the subsequent region of binocular fixation, afterre-calculation, has changed, then re-calculating the disparity range ofthe subsequent region of binocular fixation.
 12. A binocular imagingsystem, comprising: a display configured to display a current binocularimage having left and right eye views, wherein the left and right eyeviews each comprise a first portion of respective left and right eyesource images; a gaze tracker that identifies at least one or both gazedirections of the left eye and the right eye; and an image controllerconfigured to use input from the current binocular image and input froma gaze tracker to calculate a subsequent region of binocular fixation,and to determine any change in binocular fixation between a currentregion of binocular fixation and the subsequent region of binocularfixation, and to calculate a disparity range of the subsequent region ofbinocular fixation when there is a change in binocular fixation, and todetermine whether the disparity range of the subsequent region ofbinocular fixation is substantially zero, and to choose new left andright eye views to create a subsequently displayed binocular image whenthe disparity range is not substantially zero, wherein a nearest objectin the subsequent region of binocular fixation has a disparity ofsubstantially zero in the subsequently displayed binocular image, andwherein the new left and right eye views each comprise a second portionof the respective left and right eye source images, wherein the firstand second portions of the left eye source image are different portions,and the first and second portions of the right eye source image aredifferent portions, and to make the currently displayed image, thesubsequently displayed binocular image.
 13. The binocular imaging systemof claim 12, wherein the image controller is configured to receive asecond input from the gaze tracker and using the second input from thecurrent binocular image and the gaze tracker to re-calculate thesubsequent region of binocular fixation when the subsequent region ofbinocular fixation has not substantially changed, and the imagecontroller is configured to calculate the disparity range of thesubsequent region of binocular fixation, if it is determined that thesubsequent region of binocular fixation, after re-calculation, haschanged.
 14. The binocular imaging system of claim 13, wherein the imagecontroller is configured to receive a third input from the gaze trackerand using the third input from the current binocular image and the gazetracker to further re-calculate the subsequent region of binocularfixation when the disparity range is approximately zero, and the imagecontroller is configured to re-calculate the disparity range of thesubsequent region of binocular fixation, if it is determined that thesubsequent region of binocular fixation, after further re-calculation,has changed.
 15. The binocular imaging system of claim 12, wherein theimage controller is configured to receive a second input from the gazetracker and using the second input from the current binocular image andthe gaze tracker to re-calculate the subsequent region of binocularfixation when the disparity range is approximately zero, and the imagecontroller is configured to re-calculate the disparity range of thesubsequent region of binocular fixation, if it is determined that thesubsequent region of binocular fixation, after re-calculation, haschanged.
 16. The binocular imaging system of claim 12, wherein the gazetracker is configured to determine the disparity within the fixatedregion and the plane of fixation from the difference between left eyeand right eye fixation points.
 17. The binocular imaging system of claim12, wherein the image controller is configured to determine whether thedisparity range is substantially zero by comparing the image disparityof the subsequent object with zero, wherein the subsequent object isbeing imaged where it is the closest object to a viewer in the region ofbinocular fixation.
 18. The binocular imaging system of claim 12,wherein the image controller is configured to form a currently displayedbinocular image by estimating a 3D region of fixation and projecting the3D region of fixation into an image plane to form a binocular region offixation.